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INTRODUCTION 


As demand for information on postsecondary education has increased, institutions 
have had to adapt information systems to accurately report data. Institutions currently 
respond to multiple constituencies including national media outlets, higher education 
associations, state regulatory authorities, and federal agencies. The purpose of this study is to 
examine the information/data systems postsecondary institutions use to report accurate data 
and how these systems increase or decrease the burden of reporting data. Specifically this 
study examines how information systems assist institutions in reporting data to the Integrated 
Postsecondary Education Data System (IPEDS), the primary source of data on colleges, 
universities, and technical/vocational postsecondary institutions collected by the federal 


government. 


THE INTEGRATED POSTSECONDARY EDUCATION DATA SYSTEM (IPEDS) 


The Integrated Postsecondary Education Data System is a data collection and 
dissemination program managed by the National Center for Education Statistics (NCES) in 
the U.S. Department of Education (DOE). IPEDS is made up of a system of interrelated 
surveys conducted annually by NCES. The surveys collect data on each institution’s 
characteristics, institutional prices, enrollment, student/staff race/ethnicity, degrees and 
certificates conferred, student financial aid, institutional finances, student persistence and 
success, faculty salaries, and staffing levels. Reporting IPEDS data is a requirement for all 
postsecondary institutions that are eligible to receive Title IV funding through the U.S. DOE. 
Because IPEDS data collection gathers institution-level data on a variety of metrics, is 
mandatory for the majority of institutions, and is available to anyone with access to the 


Internet, IPEDS data are a rich source of information on postsecondary education. 


NCES provides tools to support review and analysis of IPEDS data from institutions. 


Examples include: 


e NCES’s online College Navigator makes IPEDS data available for to the 
general public, primarily for college choice and decision-making by 


perspective students and parents. 


e The IPEDS Data Feedback Report is a published report sent to institutions 
that compares the institution’s IPEDS data to an institutionally defined 


comparison group. 


e NCES publications such as the Digest of Education Statistics and IPEDS 
First Look provide summary information on postsecondary education. These 
publications serve as a trusted source of information on postsecondary 


education for policymakers, researchers, and the general public. 


e The IPEDS Data Center enhances the capacity of institutions to compare 
themselves with peer institutions online or by downloading datasets for 


comparison. 


e The IPEDS Data Center also assists in aggregating and analyzing IPEDS 
data from various institutions. Data are used by state legislatures, governing 
boards, Congress, and other constituencies to inform decisions about 


postsecondary education. 


Since IPEDS data are used by a variety of people and venues and for a variety of 
reasons and purposes, its accuracy is vital. The responsibility for providing accurate data to 
IPEDS is the IPEDS keyholder. 


Each institution is designated with one keyholder to coordinate data reporting for the 
postsecondary institution. Keyholders are responsible for entering data, ensuring the data are 
accurate, and locking the surveys when completed’. The IPEDS keyholder is typically an 
institutional research professional (e.g., director, assistant director, or staff within the 
institutional research department), a registrar, or an individual responsible for the 
administration of the institution. The keyholder may not have access to all needed IPEDS 
data and may need to gather data, so they depend on data systems to capture information 
from other departments within the institution. As a result, reporting to IPEDS often requires a 


great deal of communication and coordination by the keyholder. 


' Keyholders may delegate reporting by allowing other institutional representatives report data to the 
IPEDS data collection system. 


THE ROLE OF DATA SYSTEMS 


The process of reporting to IPEDS begins with the collection and storage of 
institutional data. Most institutions use campus data systems to store, update, and maintain 
institutional data, though some smaller institutions use paper-systems without a defined 
computer system in place. Many commercially available data system products have been 
specifically designed for institutions of higher education. However, institutions are not 
obligated to employ a commercially developed data system to manage their campus data, and 
some may use off-the-shelf database and statistical programs. Other institutions may develop 
their own institution-based system using Oracle, SQL, Microsoft Access, open source, or 
some other software product to fit their needs and resources.” These systems may be more 
cost-effective as well, but may also require a great deal of information technology (IT) 


support. 


Although IPEDS data can be gathered using numerous software mechanisms, 
commercial campus data systems promise to automate reporting to IPEDS by promoting 
simple input of data, storage, and reporting options. Many products also offer an “IPEDS 
report” that promises to gather, aggregate, and present all needed data to report to IPEDS. 
The ability to automatically generate IPEDS-specific reports is a central selling point for 
these systems because it centralizes data from multiple sources across an institution and 
would eliminate the need for keyholders to gather and work with raw data from many 
departments. Because these data system products vary in their complexity and utility, it is 
also helpful to understand differences in burden to keyholders using different systems. The 
need to better understand these issues led to the development of research questions listed 


below. 


* Because institutions that are part of larger systems (e.g., each campus of the State University of New York 
system) report independently, systems may be developed by the system, rather than the individual campus. 
For the purposes of this study, both are considered institution developed systems. 


RESEARCH QUESTIONS 


Little is known about how keyholders gather data in preparation for reporting to 


IPEDS, including the information data systems they use, perceptions of those systems, and 


using these systems to generate reports for IPEDS reporting. The following research 


questions guide this study: 


1. What types of campus data systems are institutions of higher education using to assist in 
completing IPEDS surveys? 


2. What are the features of commercially available campus data systems commonly used by 
institutions of higher education? 


3. In what ways do campus data systems and their features affect keyholder burden in 
completing IPEDS surveys? 


BRIEF METHODOLOGY 


To address this project’s research questions, three data collection strategies were 


implemented between July 2008 and June 2009. Data collection for this study consisted of a 


keyholder survey, interviews with a subset of keyholders, and online demonstrations of 


products and correspondence with product users, as described below: 


iF 


The Keyholder Survey. The keyholder survey was a web-based survey of 729 
potential’ IPEDS keyholders at 4- and 2-year degree-granting institutions across 


the United States. 141 participants submitted complete surveys resulting in a 19% 
response rate. 


Keyholder survey items were based on the literature demonstrating high 
keyholder turnover in institutional research (IR), suggesting that keyholders’ 
experience levels in IR varied greatly (IPEDS Web-based Data Collection 
System, 2009) and were composed to address questions about the nature of 
campus data systems used by institutions. Respondents reported on their 
experience in two items. The first item asked respondents, “How many years 
have you held your current position?” The second item asked respondents, “How 


* Respondents were identified by targeting individuals with job titles commonly associated with 
involvement in IPEDS data collections, including institutional research professionals, registrars, and budget 
and planning specialists. 


long have you worked in the IR field?” Survey items also asked respondents to 
indicate which campus data system was used on their campus, the features of that 
system, and what features they thought would help improve the process of 
reporting to IPEDS. 


Additional survey items were based on the need to determine how keyholders 
perceive their burden. Feelings of burden have been shown to influence 
motivation to perform a task (such as reporting to IPEDS) and are thought to be a 
key factor in the individual’s longevity in the IR field (Deci & Ryan, 2000; 
Pekrun et al., 2002; Pekrun, 2006). Other survey items asked respondents to 
indicate their thoughts and feelings concerning the campus data system and 
reporting to IPEDS. 


2. Keyholder Interviews. A subset of survey participants (n = 11) were selected for 
participation in interviews. Potential interview participants were purposefully 


selected to include a range of institution control and location (1.e., region) and by 
the participant’s willingness to be interviewed. Interview participants were 
contacted via e-mail after the second month of survey collections to confirm their 
willingness to be interviewed and obtain their consent to participate in the 
interview. 


3. Product Demonstration. Online demonstrations of campus data system products, 
plus correspondence with sales representatives and references from vendors, 


provided further clarification of the features of each campus data system product. 
Statements from product vendors are presented separately from the survey and 
interview results in the following sections. 

Respondents to the web survey came from institutions from a range of sectors and 
sizes. They represent the range of the university professional community, including 
specialists in institutional research, department heads (e.g., human resources), and other 
research and information technology positions. Characteristics of the survey sample are 


shown in table 1. 


Participants in personal interviews were also drawn from across postsecondary 
institution sectors and had varying professional positions within their institutions. Only one 
interview participant self-identified as an institutional research professional, while seven 
identified their positions as other management positions within the institution (e.g., director 
of student services, admissions, and financial aid). The remaining three participants worked 


within information technology or served in research positions at their institution. 


Table 1. Characteristics of survey respondents and their institutions 


Characteristics of institution and respondents n Percent 


Total 141 100 


Institution type 


Public 4-year 22 15.6 
Public 2-year 49 34.8 
Private not for profit, no religious affiliation 31 22.0 
Private not for profit, with religious affiliation 25 17.7 
Private for profit 14 9.9 


Institution size 


Less than 1,000 students 40 28.4 
1,000-2,500 students 28 19.9 
2,501—5,000 students 27 19.1 
5,001-10,000 students 17 12.1 
10,001-15,000 students 10 7.1 
More than 15,000 students’ 19 13.5 


Respondent current position 


Institutional research (director, asst. V.P., etc.) 62 44.0 
Department head (e.g., financial aid, registrar) 34 24.1 
Owner, CEO, president 5 3.5 
Other management position 11 78 
Research associate or professional 12 8.5 
Research or administrative assistant 12 8.5 
Not reported 5 3.5 


Respondent time in current position 


Less than 1 year 16 11.3 
1-3 years 33 23.4 
4-7 years 34 24.1 
8-10 years 19 13.5 
More than 10 years 39 27.7 


1 This category is the sum of responses for institution sizes 15,001—20,000, 20,001-30,000, and more 
than 30,000 students, each of which had fewer than 10 respondents. 


FINDINGS: DATA SYSTEMS USED BY SURVEY RESPONDENTS 


Survey respondents were asked to indicate the data system or software product they 
used to collect data across a range of institution functions. These included administrative 
data, academic records, billing data, student financial aid, student registration, institution 
advancement, and course management. Respondents were also asked to indicate the number 


of years they have used their current system. 


Responses on the open-ended items asking respondents to identify data systems 
resulted in a broad range of responses. Several widely known products were identified (e.g., 
Banner, Datatel), while a number of systems were reported by only one or two respondents. 
Unless these products represented specific institutional research niches (e.g., Raiser’s Edge 
for institutional advancement), they were grouped together as “other software.” For each 
campus data collection purpose, respondents indicated the use of off-the-shelf software not 
specifically designed for educational data management (e.g., Microsoft Excel, QuickBooks). 
Although not designed specifically for institutional research purposes, these products have 
wide-scale use and available technical support and are generally accessible. These were 
categorized as “repurposed off-the-shelf” systems. Some respondents reported the use of 
internally developed, institution-based systems, including systems that were designed 
specifically for their campus or for a larger university system of which their campus is an 
affiliate. These systems may be unknown outside of their local institution, but they may also 
offer the advantage of being customized to meet local research and reporting needs. Finally, 
some respondents indicated that they did not use a computerized system, relying on “hand 


count” to determine the data necessary for reporting to IPEDS. 


Respondents were also asked how long they had been using their current system. 
These data are important for several reasons. First, the respondent’s experience with a system 
likely increases his or her confidence in the system and perception of ease of use, making 
reporting a positive, rather than negative, responsibility for the keyholder. Second, the 
duration of use of each system is of interest because of changes implemented in the IPEDS 
system in 2002. Institutional response to these changes may have resulted in a change in local 
campus data system choices—so examining systems used only since the IPEDS 


modifications and those that were used before and after the change may inform institutional 


Table 2. Percentage of respondents reporting use of various systems to provide data to IPEDS and time using current system, by type of data reported (N = 141) 


Administrative Academic Financial aid Advancement Course Registration 
data records data Billing data data management data 
System used: 

Banner 19.1 20.6 19.9 19.9 9.9 17.0 20.6 
Datatel 12.8 13.5 12.8 13.5 6.4 11.3 13.5 
Jenzabar 12.1 13.5 tA 12.8 8.5 8.5 13.5 
PeopleSoft 7.8 0.0 8.5 7.8 5.7 5.0 7.8 
PowerCAMPUS 28 2.8 1.4 2.8 2.1 14 4.3 
CAMS 1.4 1.4 14 1.4 0.7 1.4 1.4 
Raiser’s Edge/Blackbaud 0.7 0.7 0.0 0.7 9.2 0.0 0.7 
PowerFAIDS 0.0 0.0 7 0.0 0.0 0.0 0.0 
Blackboard 0.0 0.0 0.0 0.0 0.0 5.7 0.0 
Recruitment Plus 43 0.0 0.0 0.0 0.0 0.0 0.0 
Repurposed off-the-shelf 2.8 2.1 2.8 5.0 0.7 1.4 3.5 
All other products 17.8 25.5 18.5 14.1 10 22 14.8 
Institution developed 3.5 5.0 30 5.7 2.8 43 5.7 
None 0.7 0.7 0.0 0.0 4.3 2.1 0.7 
Not reported 14.2 14.2 17.0 16.3 39.7 19.9 13.5 

Time using current system: 
Less than 1 year 2.1 2.8 2.1 2.1 43 2.8 2.1 
1-3 years 16.3 12.1 9.9 14.2 13.5 13.5 13.5 
4-7 years 18.4 17.0 19.1 15.6 12.1 18.4 17.0 
8-10 years 17.7 17.0 17.0 17.7 8.5 14.2 17.7 
11-20 years 14.2 15.6 14.2 13.5 78 8.5 14.2 
More than 20 years 28 5.0 3.5 3.5 2.8 43 5.7 
Don’t know 0.0 0.0 0.7 0.0 2.1 2.1 0.0 
Not reported 28.4 30.5 33.3 33.3 48.9 36.2 29.8 


choices. Data systems identified by respondents across campus departments and their duration of use 


are shown in table 2. 


The most commonly reported data systems across purposes were Banner, Datatel, and 
Jenzabar, although there was a very high rate of missing data for advancement and course 
management data. Other systems were reported primarily for specific purposes, reflecting their 
intentional design. Systems such as PowerFAIDS for financial aid services, Raiser’s Edge for 
institutional advancement, Recruitment PLUS for administration data, and Blackboard for course 
management were also frequently cited for their specific purposes but for no others. The percentage 
of institutions using repurposed software and institution-developed systems was generally low (about 
5 percent or less across systems), and reports of no computer system in use were very rare. However, 
reports of missing data (14-17 percent for most data needs, but as high as 39.7 percent for 
advancement data and 19.1 percent for course management data) suggest that systems may not be in 
place, or that respondents lack knowledge of them. In addition, the “other software” category, which 
in some domains was the plurality report, captured as many as 25 to 30 different software products 
identified. This item was an open-ended response, and some of abbreviations could not be easily or 
readily reconciled and may represent institution-developed systems. Data collected from the survey 


and subsequent interviews do not help identify all of the software products used. 


The duration of use of current campus data systems suggests some differences in the speed 
with which institutions adopt new systems. Few respondents indicated that their current systems had 
been in place for less than a year. However, some data systems appear to be earlier in their adoption 
phase than others. For example, systems for advancement data (29.9 percent in use 7 years or less) 
and course management (34.7 percent in use 7 years or less) appear to be in their early adoption 
stage, while systems for all other types of data are more evenly distributed in duration of use. 
Interestingly, a small percentage of respondents reported the use of the same data system for 20 or 
more years. The duration of use also indicates institutional commitment to its chosen system. More 
than half of respondents (57.8 percent) to the survey indicated that their institutions did not have 
plans to change or upgrade their current systems “in the near future.” About one in four (24.4 
percent) indicated that their institution did have such changes planned, while 17.8 percent indicated 


that they did not know of their institution’s future plans. 


These survey results were augmented by comments from interview participants. Many 


institutions may have outside contractors or data managers (as opposed to someone on site) to handle 


data reporting. Perhaps employees at those institutions had more difficulty in providing the 
information needed to complete the survey and so left responses blank. Those institutions that use 
outside contractors or data managers may be more likely to use institution-developed or Microsoft 
Office systems as well since many of those institutions are smaller and the data may be easier to 


manage with a simpler, less expensive product. 


FINDINGS: REVIEW OF CAMPUS INFORMATION SYSTEM TOOLS AND THEIR 
UTILIZATION 


This study examined common campus information systems through a review of the most 
commonly used systems’ documentation, and when possible, interviews with their publishers. In 
addition, the survey asked respondents about the availability of special tools within campus data 


systems intended to facilitate IPEDS reporting and their utility. 


Review of Campus Information System Tools 

There are many different campus data systems on the market, many of which are more 
appropriate for an institution of a certain institutional size and scope. Most campus data systems can 
be tailored to match any type of institution, but large and complex campus data systems are 
expensive and are often used by larger institutions. The survey results indicated that four main 
campus data system products are used by the majority of institutions of higher education: Banner, 
Colleague (Datatel), Jenzabar, and PeopleSoft. Campus data system product vendors identified in the 
respondent survey were contacted concerning the features of the product they offered institutions. 
Some vendors were helpful in providing detailed information concerning the product, while others 
directed us to the product website for information. Table 3 shows some of the campus data systems 
offered to institutions of higher education and the features of those systems as reported by the 


product vendors. 
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Table 3. Features of commonly used campus data systems 


Products and Features Banner: CAMS Three Datatel PeopleSoft: _PowerCAMPUS: 
Sungard Rivers Colleague § Jenzabar Oracle Sungard 
Suite includes all major parts Yes Yes Yes Yes Yes Yes 
Microsoft-compliant reporting Yes Yes Yes: can also Yes Yes Yes 
run SQL 

Standard reports to include IPEDS Yes Yes (plus BYOR Yes: both ST Yes Yes Yes 

makes it easy) and HR 
Query reports with ease Yes Yes Yes Yes Yes Yes 
Can share all reports online Yes Yes Yes Yes Yes Some client- 

based 

Real-time updates Yes Yes Yes Yes Yes Yes 
Fully integrated modules Yes Yes Yes Yes Yes Yes 
Notes. 


Suite includes admissions, academic records, billing and cash receipts, advancement, financial aid, online advising, online reports, 
online grading, online course management, online access to students, online registration, online payment and account information, 
and online access to alumni. BYOR: Build your own report (report writer). 


Survey Respondent’s Access to and Utilization of Special Reporting Tool 


Taken as a whole, commonly used data systems appear to provide tools that should support 


institutional researchers’ reporting to IPEDS. However, this conclusion is based upon marketing 


materials and interactions with vendors and does little to address the degree to which these tools are 


actually useful to IPEDS keyholders. Therefore, survey respondents were asked to indicate the 


degree to which their campus data system has features that should make reporting to IPEDS easier, 


as well as the degree to which they felt their campus data systems had features that actually made 


reporting to IPEDS easier. Their responses to these two items are shown in table 4. Note that 


responses to these items did not vary by characteristics of the respondent and the institution (the 


distribution of campus data systems in use did not allow for testing differences by system in use). 
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Table 4. Percentage of respondents who reported of the presence of campus data system features that should make 
reporting to IPEDS easier, features that do make reporting to IPEDS easier, and the helpfulness of these features 


Not at all Somewhat Mostly Extremely No response 

Campus data system has features that should 13.5 23.4 29.8 12.8 20.6 
make reporting to IPEDS easier 

Campus data system has features that make 24.1 34.8 17.7 43 19.1 
reporting to IPEDS easier 

How helpful are these features when reporting 25.5 31.2 17.7 28 22.1 
to IPEDS? 

Notes. 


Number of respondents = 141. 


As shown in table 4, IPEDS keyholders generally felt that their campus data system includes 
features that should provide some support for IPEDS reporting, but their views on the degree to 
which these features actually were helpful were less enthusiastic. Keyholder views of the usefulness 
of campus data system features for reporting to IPEDS was highly correlated with their reported 
satisfaction with the system (r = .38, p < .01). When asked about their overall satisfaction with their 
current campus data system when reporting to IPEDS, more than half (51.8 percent) of respondents 
were mostly or extremely satisfied with their system, while 28.4 percent were somewhat satisfied, 
and 9.2 percent were not at all satisfied with their current system (an additional 10.7 percent 
indicated it was “too early to tell” or did not respond to this item). However, the high level of missing 
data on these items makes a clear interpretation more challenging. The utility of available campus 
data system features was therefore further explored through direct interviews of 11 IPEDS 


keyholders. 


Interview Findings About Campus Data System Ready Reporting Use 

All of the interview participants indicated that they did not make use of the IPEDS ready- 
made report offered by the campus data system, but instead used other software, created unique 
queries, or used some other combination of actions to gather data to report to IPEDS. Their 
comments about these reports demonstrated that the keyholder’s technical skills development (or 
lack thereof) and his or her opinion of the campus data system contributed to how much the 
keyholder used the features of the campus data system to complete the IPEDS surveys. An overall 
theme found in the interview results is that even though the campus data system products claim to 


offer ready-made reports to gather needed data for IPEDS, the keyholder often has to create his or 
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her own reports or manipulate data in another way in order to gather the data needed to complete the 


IPEDS surveys. 


When asked about access to data and extracting data from the campus data systems, many 
institutional research professionals said that they do not use the canned reports available through the 
campus data systems but rather prefer to use other programming software or “raw data” to acquire 
the information needed to complete reports such as IPEDS. One institutional research professional 
said that he “work[s] directly with the tables, I don’t use the [data system] interface. I use Microsoft 
Access and with the SQL programming I can read right off of the tables. They have given me access 
to a few of the [data system] interface screens but they aren’t useful for my purposes.” He went on to 
say that he has “...a good deal of a programming background. I can program in several different 
languages, so I am comfortable with doing that.” Numerous interview participants made similar 
comments. For example, one participant said the campus data system on his campus 

...has them [canned reports] but I don’t use them. I am just more of a hands- 
on [person], I would rather write my own, I am more ‘you tell me the data 
you want and I know where it’s at, and I'll go and get it.’ And I am more 
comfortable with that and like to do it that way. Other people would say there 
are some things in [the data system] that will pull some of your IPEDS data 
but, uh, I like to look at the data as I pull it. And it’s a game to me. I look at, 
and I expect my data to look like this and I pull it and hopefully...if it’s close I 


am happy and if it’s not I say, ‘Whoops, let’s look at this again, let’s look at 
why,’ 


Other interview participants said that a programming background has assisted in their success 
as an institutional research professional. One such participant said that relying on the campus data 
system canned reports is not ideal because “...I didn’t write that code so I can’t really say for sure 
that I know what it is pulling. When I use my Impromptu and [programming software] reports I 
know what I am pulling, I know what I want after and I know what I am using in that report. And it 
makes me feel a whole lot better.” Likewise, another participant noted a preference for extracting 
data directly, “Because rather than having multiple reports to pull all of the information out of, I 
build a report that can be run from a [programming software] into a data table. ...i1t’s easier to get the 


information we are looking for rather than using the canned reports that are available.” 


Competent Technical Skills: Data Warehouse 
Many interview participants reported that they have access to a data warehouse on their 
campus. A data warehouse is another source for data commonly used by IR professionals that forms 


a fixed dataset (not live, or changing), which makes analyses of the data stable and consistent. One 
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interview participant noted that “...we have a system...called data warehouse. So I can go in there 


and run queries if I need to. So, there is a lot of information available there.” 


Several IR professionals reported using Cognos software to access data within the data 
warehouse as a way to work with the raw data as opposed to relying on canned reports available in 
the campus data system. One participant noted that “my staff [and I] ... spend much more of our time 
in Cognos as a reporting tool than we do in the campus data system. [It] is more of a transaction 


system. And then we use Cognos as the reporting.” He went on to say that the campus data system: 


..is very confusing...it is to me. But it is very challenging, unless you are a 
pure programmer kind of person. So interfacing to the warehouse turns it into 
data that me and a large number of end users on campus can touch and you 
know we don’t have to understand strange language and bizarre field 
names... So that is why Iam glad we have Cognos. Because without 
Cognos...like any system the canned reports are not exactly what the person 
wants. And they want them tweaked a little bit and that is difficult. So...if it is 
something that doesn’t fit well with something already in the system, we can 
always go to Cognos and pull a custom report... So, if you rely on the campus 
data system you have to rely on their programmers to make the change in 
their software. So, with Cognos we have some more flexibility. 


Lack of Technical Skills 


Unfortunately, many IR professionals do not have the technical skills needed to overcome the 
faults of the campus data system and the data access issues that may exist on their campus. One 
participant expressed his frustration with a lack of technical skills that the prior IR professional at the 


institution had, stating that 


...the problem stems from when I got here...I think it is that the prior person 
had query experience with InfoMaker and could go out there and query 
information. And other people [on campus] had query information so they 
would report numbers and all the numbers were different, because it 
depended on how they wrote the query and how they specified it. That was 
one of the things that I was asked to fix. And the only way you can fix it is by 
having it restricted. Where this office reports the information and I am not 
responsible for how the query is formed because I am not a database 
administrator and I don’t know how their system all works. I just ask for the 
information and they [IT] give it to me. 


Many survey and interview respondents reported that they do not use the canned reports 
available through the campus data systems. Interviews with IR professionals revealed similar 


findings. One participant stated that she 
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... [hasn't] looked at [the canned reports] since 2000 because when we looked 
at it then, it really wasn’t doing what they [the vendor] claimed it did, it 
didn’t do your IPEDS reports ...so we really haven't looked at it since then. 
We have our own files that we create from our census date. We freeze the 
entire data base. And then we create our own files that we use to put our 
various reports together. And one of the biggest things we have is just 
cleaning up the data. So that’s what we use for our IPEDS reports because 
we know that that data is clean. 


The participant mentioned during several phases of the interview the importance of reporting 
clean data when reporting institutional data and believed that the campus data system canned reports 


did not always produce clean data for reporting. 


Others had similar concerns: “There [are] a lot of canned reports, however I use none of 
them!” Some of the reasons for not using the campus data system canned reports were being unsure 
of the results, the reports not providing the needed results, and the reports being “unreliable” and 


“not up to date with current changes to IPEDS.” 


As a result, many keyholders turn to other technical software (e.g., InfoMaker, Cognos with 
data warehouse) to pull needed data for reporting, rather than rely upon campus data system reports. 
Many respondents mentioned that they have customized the campus data system reports to pull the 


needed data but did not use the canned reports as delivered from the manufacturer. 


FINDINGS: THE SURVEY RESPONDENTS’ ATTITUDES TOWARD CAMPUS DATA 
SYSTEMS AND IPEDS REPORTING 


Survey respondents were asked a series of questions about their attitudes toward the campus 
data systems in use at their institutions and their attitudes towards IPEDS reporting. It is likely that 
IR professionals, who are more comfortable and confident in the use of their campus data system, 


may be more efficient at completing their IPEDS reports and may also provide more accurate data. 


Survey Respondents’ Attitudes About the Campus Data System in Use 
Survey respondents were asked the degree to which their campus data systems evoked 
specific emotional reactions, including positive reactions (e.g., contentment, happiness) and negative 


reactions (e.g., anxiety, anger). Their responses to these items are shown in table 5. 
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Table 5. Respondents’ views about their current campus data system 


Item description Response option n Percent 
When | think about my campus data system, | feel angry. A little bit 66 46.8 
Somewhat 25 17.7 
Quite a bit 17 12.1 
Very much ] 6.4 
Extremely 5 3.5 
No response 19 13.5 
When | think about my campus data system, | feel anxious. A little bit 28 19.9 
Somewhat 31 22.0 
Quite a bit 36 25.5 
Very much 16 11.3 
Extremely 9g 6.4 
No response 21 14.9 
When | think about my campus data system, | feel confident A little bit ] 6.4 
Somewhat 14 9.9 
Quite a bit 30 21.3 
Very much 21 14.9 
Extremely 41 29.1 
No response 26 18.4 
When | think about my campus data system, | feel content. A little bit 13 9.2 
Somewhat 12 8.5 
Quite a bit 33 23.4 
Very much 32 22.7 
Extremely 29 20.6 
No response 22 15.6 
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Item description Response option n Percent 
When | think about my campus data system, | feel determined. A little bit 17 12.1 
Somewhat 11 7.8 
Quite a bit 32 22.7 
Very much 31 22.0 
Extremely 25 17.7 
No response 25 17.7 
When | think about my campus data system, | feel happy. A little bit 22 15.6 
Somewhat 23 16.3 
Quite a bit 28 19.9 
Very much 30 21.3 
Extremely 18 12.8 
No response 20 14.2 
When | think about my campus data system, | feel insecure. A little bit 41 29.1 
Somewhat 28 19.9 
Quite a bit 30 21.3 
Very much 14 9.9 
Extremely 6 4.3 
No response 22 15.6 
When | think about my campus data system, | feel A little bit 42 29.8 
overwhelmed. Somewhat 32 22.7 
Quite a bit 24 17.0 
Very much 12 8.5 
Extremely 10 7.1 
No response 21 14.9 
When | think about my campus data system, | feel reluctant. A little bit 57 40.4 
Somewhat 27 19.1 
Quite a bit 24 17.0 
Very much 13 9.2 
Extremely 3 2.1 
No response 17 12.1 
When | think about my campus data system, | feel satisfied. A little bit 13 9.2 
Somewhat 17 12.1 
Quite a bit 31 22.0 
Very much 24 17.0 
Extremely 36 25.5 
No response 20 14.2 
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Survey Respondents’ Attitudes About IPEDS Reporting 

Survey respondents were also asked questions specifically about their role as IPEDS 
keyholders and their views of completing IPEDS surveys. Specific to their roles as IPEDS 
keyholders, respondents were asked about the degree to which they perceived IPEDS reporting to be 
a burden, as well as a major task within their assigned job duties. Their responses are shown in table 
6. The perception of burden was correlated with time spent (7 = .59, p < .01), but not importance (7 = 


.02, ns). 


Table 6. Respondent perception of the importance of IPEDS reporting and its burden (n= 141) 


Not at all Somewhat Mostly Extremely No response 
IPEDS is a burdensome part of my job 25.5 37.6 17.0 7.1 12.8 
IPEDS is a major task (in terms of time) dea 39.7 19.9 16.3 17.1 
IPEDS is a major task (in terms of importance) 4.3 22.0 28.4 29.8 15.5 


Respondents were also asked a series of questions about their emotional responses to their 
role as IPEDS keyholders. These items were comparable to those asked about their campus data 


systems (see table 5). Responses to these items are shown in table 7. 


Table 7. Respondents’ views about reporting to IPEDS 


Item description Response option n Percent 
When | think about reporting to IPEDS, | feel angry. A little bit 82 58.2 
Somewhat 12 8.5 
Quite a bit 12 8.5 
Very much 6 43 
Extremely 2 1.4 
No response 27 19.1 
When | think about reporting to IPEDS, | feel anxious. A little bit 26 18.4 
Somewhat 28 19.9 
Quite a bit 27 19.1 
Very much 18 12.8 
Extremely 13 9.2 
No response 29 20.6 
When | think about reporting to IPEDS, | feel content. A little bit 21 14.9 
Somewhat 20 14.2 
Quite a bit 34 24.1 
Very much 24 17.0 
Extremely 16 11.3 
No response 26 18.4 
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Item description Response option n Percent 


When | think about reporting to IPEDS, | feel determined. A little bit 22 15.6 
Somewhat 10 7.1 
Quite a bit 28 19.9 
Very much 27 19.1 
Extremely 19 13.5 
No response 35 24.8 
When | think about reporting to IPEDS, | feel happy. A little bit 49 34.8 
Somewhat 22 15.6 
Quite a bit 19 13.5 
Very much 24 17.0 
Extremely 4 28 
No response 23 16.3 
When | think about reporting to IPEDS, | feel insecure. A little bit 49 34.8 
Somewhat 34 24.1 
Quite a bit 18 12.8 
Very much 10 7.1 
Extremely 6 43 
No response 24 17.0 
When | think about reporting to IPEDS, | feel overwhelmed. A little bit 40 28.4 
Somewhat 26 18.4 
Quite a bit 23 16.3 
Very much 16 11.3 
Extremely 11 78 
No response 25 17.7 
When | think about reporting to IPEDS, | feel reluctant. A little bit 56 39.7 
Somewhat 32 22.7 
Quite a bit 16 11.3 
Very much 9 6.4 
Extremely 4 28 
No response 24 17.0 
When | think about reporting to IPEDS, | feel satisfied. A little bit 16 11.3 
Somewhat 21 14.9 
Quite a bit 31 22.0 
Very much 25 17.7 
Extremely 20 14.2 
No response 28 19.9 


Composites were also developed from these items to reflect overall positive responses (mean 
ratings for content, determined, happy, and satisfied) and negative responses (mean ratings for anger, 
anxiety, reluctance, insecure, and overwhelmed). These overall scores were not highly correlated (7 = 
.14, ns) with each other. Respondents’ ratings of their overall satisfaction with their current campus 


data system was correlated with their overall positive response to their campus data system (7 = .67, p 
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< .01), positive views when reporting to IPEDS (r = .24, p < .01), and less negative views of 


reporting to IPEDS (r= -.45, p< .01). 


LIMITATIONS 


This study utilized a mixed-methods approach to examine information/data systems at 
degree-granting postsecondary institutions. Using a qualitative approach limits the generalizability 
of this study to all degree-granting institutions. The low number of respondents to the survey portion 
of this study (n=141) cannot be used to represent all postsecondary institutions. Additionally, the 
breadth of responses did not allow for comparisons across institution or keyholder characteristics, but 
the responses do provide a potential starting point for additional study that can target specific 
institution types either by characteristic or campus data system employed. The strength of a mixed- 
methods approach provides readers with more indepth and rich information than is available in a 


survey study, especially when combining survey data with qualitative interviews. 


SUMMARY AND CONCLUSION 


This study was largely intended to be exploratory in nature. In this section key findings are 
summarized, and several recommendations are made regarding steps that can be taken to enhance the 


potential for IPEDS keyholders to accurately and efficiently report IPEDS data. 
Summary of Key Findings 
The survey of IPEDS keyholders and interviews revealed the following findings: 


e Institutions rely upon a range of campus data systems. The most common systems are 


Banner, Datatel, Jenzabar, and Peoplesoft. 


e While four commercially available products were most commonly used, there was no clearly 
dominant system, and institutions instead seemed to use campus data systems based upon 


their own reasons. 
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e The majority of respondents indicated generally positive feelings toward their campus data 
systems and reporting to IPEDS. They also tended to see their role as IPEDS keyholder as 
important, but also noted the high volume of work. None of these perceptions were related to 


characteristics of the respondent or institution. 


e There was a tendency for individuals specifically in the role of IR professional to view their 


systems and IPEDS responsibilities more positively. 


e Despite the availability of ready-made reports (such as IPEDS reports) in campus data 
systems, most keyholders do not use these reports to gather the data needed for IPEDS. 
Interviews were conducted to determine why the reports are not used. Many of the 
interviewees mentioned that they have developed their own reports to collect the data needed 
for IPEDS to be sure the correct data are being gathered. Interview results also demonstrated 
that despite the data systems’ ready-made reports that promise reporting to IPEDS with ease, 
the reports are often difficult to use and require numerous steps. Reports built by the 


keyholder are trusted and known to gather the correct data needed for IPEDS. 


Recommendations Based on Results 
This section provides recommendations for various groups and individuals based on the 


results of this study. 


Ready-Made Campus Data System Reports. Interviews revealed that keyholders with full access 


to the campus data system and the available reports do not tend to use the canned reports (such as 
those provided to report to IPEDS), as the reports are often inaccurate. At the same time, IR 
professionals noted the level of work required to respond to data requests. Several possibilities arise 


from this need for accurate campus data system reports: 


1. NCES could work IPEDS keyholders to develop additional tools to reduce workload 
related to IPEDS data reporting. 


2. Campus data system vendors could work with keyholders to improve the usability of the 
reports function of the data systems.. 


3. AIR trainers could train IPEDS keyholders to use the various campus data systems and 
their canned reports to assist them in reporting to IPEDS. 
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4. AIR could provide IPEDS keyholders with discussion boards or electronic mailing lists 
so the users of various campus data systems could have a central place to discuss the 
issues of reporting to IPEDS using that particular software. 


Developing Competence in Technical Skills. Another finding emanating from the interviews 


conducted within this study was the importance of keyholders’ having adequate technical skills for 
working with data and extracting the needed information to complete reporting requirements, such as 


for IPEDS. Below are two possible means of addressing this apparent need: 


1. Training can be provided to keyholders in the use of existing data management software 
(e.g., Excel, SPSS) to extract data. 


2. Training sessions on software querying tools and IT programming may also provide 
assistance to new keyholders. 


This study was undertaken to better understand how IPEDS keyholders use local campus data 
systems to complete their IPEDS reports. While these data may be based upon a modest sample, they 
are suggestive of future research needs. Specifically, there were no clear patterns in the use of 
commercially available products, especially when there are other avenues of support for institutions 
reporting to IPEDS. Likewise, the survey and interviews were of limited scope and did not directly 
tap institution motivations for using one system or another. Although we collected general 
impressions of the campus data systems and IPEDS, it may be necessary to develop more detailed 
data collection strategies to understand these decisions. Finally, although one motivation for 
understanding how IPEDS keyholders prepare IPEDS reports concern regarding the quality of data 
reported, this study did not examine data quality in any way. Perhaps future efforts can address these 


issues. 
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